Entropy-based space-time segmentation in video sequences

This paper deals with motion-based segmentation of video sequences. Classical methods are based on the minimization of a penalty function of the prediction error (e.g. robust estimators). However, these estimators may not be efficient when the statistical distribution of the prediction error is not parametric. Instead, we propose to minimize the entropy of the prediction error while approximating its distribution by a non-parametric estimate. This approach only accounts for motion and cannot perform well in homogeneous regions. Therefore, a spatial term was added in the form of the entropy of the frame intensity. By combining motion and intensity (or color), it should be possible to segment objects which cannot be segmented correctly with one of these criterions alone. Minimization of the combined energy is achieved with an active contour algorithm. This method was applied to video segmentation and tracking on a synthetic, textured sequence and real world, standard sequences. Our algorithm show good results even when occlusions occur.